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ABSTRACT 

The usual reason for using the Q-sort technique has 
been to acquire information about the subjects doing the sorting, but 
this paper concerns the construction of a Q-sort which deals with 
information about the items comprising the Q-sort; more specifically, 
the knowledge and ability competencies of teachers of graduate level 
courses. A list of competency items was compiled from the state 
requirements for certification, the course descriptions in the 
university catalog, the faculty involved, and a critical incident 
questionnaire completed by a sample o2 students. The items were 
placed on the backs of cards; students were asked to sort them as to 
relative importance; and teachers were asked to sort them using their 
own level of adequacy as the criterion. In this way, the level of 
adequacy for each competency for each teacher can be determined and 
compared with the results of the student data, identifying gaps 
between what the students feel is important for the faculty to be 
competent in and what the faculty feel they are actually competent 
in. (BW) 
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A Uss of the Q-sort Technique in Educational EvaluRtion 

There are various instruments which can effectively be 
used to evaluate an educational program. On© practical and in- 
formative instrument which has been used is the Q-aort technique. 
The usual reason for using the Q-sort technique has been to acquire 
. Information, about the subjects doing the sortliig rather than 
information about the items comprising the Q-sort. Many of the 
studies which used a Q-sort technicue such as Stephenson(1953,1967 ) , 
Kerlinger(1972), and Broen(1957) were basically attempting to 
classify groups of persons according to their responses to the 
items. There have been relatively few studies which dealt with 
the items of the Q-sort itself and the information acquired about 
them. One of those that did was a study by Sontag(1968) in which 
the task was to determine which behaviors of elementary and sec- 
ondary school teachers would be rated as desirable. A Q-sort of 
items which were brief descriptions of teaching behaviors waa 
presented to a large sample of elementary and secondary teachers. 
The teachers then rank ordered the items according to desirability. 
Now that it was determined by consensus of this sample which 
behaviors were desirable, present and incoming teachers could 
be evaluated according to this established criteria. Another 
example of using a Q-sort to acquire information about the items 
of the Q-sort was a stiidy by the Far West Laboratory For Educational 
Research and Development ( 1973 ) . This Q-sort was sent through the 
mail to various personel associated with Early Childhood Education 
or Head Start programs. It asked that various competencies of 
teachers in child develooment programs be ranked in order of 
importance. From this general survey, the most important com- 
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petencies could be determined and Incorporated into the training 
prop;rams for teachers in this field. The. particular use of the 
Q-sort to be explained follows close to the Far West Laboratory 
study since information about competencies wishes to be acquired 
rather th^n information about the sample responding to the instru- 
ment. More specifically, this paper concerns the construction 
of a q-sort which deals with the knowledge and ability competencies 
of teachers of graduate level courses. Through this techniaue 
it is possible to determine the amount of importance that graduate 
students place upon certain competencies of a graduate level teacher. 

By again using this same technique, it c&n be detemined in 
which areas the actual teachers concerned fe:sl thet they have 
their strongest competencies. The function of educational evaluation 
is achieved by comparing the competencies rated important by the 
students to the actual competencies possessed by their teachers. 
For areas of expertise that the students feel are mportant. 
It can be determined if there exists among the facul a person 
or persons who feel the adequately possess this expertise. Gaps 
between what competencies the students feel are important for 
the teacher to have and what the teachers actually have could 
be identified. From such identifications, .recomendations couM be 
made as to how the curriculum and/or the faculty could be changed 
to fill these gaps. For any given program, therfore, its faculty 
could be evaluated by this technique in terras. of their fufilling 
the areas of competency considered important by the students , 

This Q-sort technique was incorporated into a larger eval- 
uation design of the Master's program of ^ small, private, 
graduate institution. In this institution the Master's program 
vaa subdivided into four separate programs (Counseling and 
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Guidance, Early Childhood Educntion, Educational Administration 
and Supervision, and Learning Technology).. It wes necessary, 
therefore, to evaluate each program separately according to the 
specific competencies reouired for teaching that field or specialty. 

« 

Method of Construction 

The first step was a method for determining the list of 
competency items for the Q-sorts in each specific program. Lists 
of competencies needed to be fomed which were appropriate for a 
specific program but would obviously differ between programs. In 
other words, no general list could be formed because many competencie 
were specific to certain programs. There were four sources of 
Information which determined the list of teacher competencies for 
any given program. One basic source was the state recuirements 
of competency for certification. This was only applicable to those 
programs related to state certification. Another basic source 
was the course descriptions given in the university's catalogue. 
Prom these detailed descriptions it was easy to derive certain 
competencies that a person teaching such a course must possess. 
Thses course descriptions also brought to light both the practical 
(things a teacher should know how to do) and the theoretical or 
knowledge (things a teacher should know about) competencies for 
each program. A third source was the faculty involved. Each 
teacher was recuested to add on to the lists of competencies 
acouired from the first two sources. In this way certain specific 
competencies not otherwise revealed would be included. It would 
have been better if a large survey similar to the Ftir West Labo:- 
atory survey could have been made of several teachers in each field, 
so that a truly comprehensive list of competencies might be formed.'" 
Due to time and .economic limitations this survey was not attempted. 



Therefore, only the additions (and biases) of the faculty involved 
were Included. A fourth source which also revealed specific 
competencie=s was a critical incident ouestionnaire completed by 
a sample of students. Prom the cril Leal incident responses, 
both positive and negative, additon. OT--jtencles could be 
,. recognized and included in the lists. The combination of these 
sources created reasonable lists of competencies for a teacher 
in each of the four specific graduate programs .( see Appendix A). 

The items on the lists were transfered by means of a computer 
to adhesive lables which were placed on the back of individual 
computer cards. Each card was rar, mly numbered and the set of 
containing all the items of a particular list were combined 
into a deck. These Q-sort decks were then administered in a group 
setting to several graduate classes. Each student recieveda leek 
which contained only the items pertaining to his particular program 
(i.e. Counseling and Guidance), a list of instructions, and a 
score sheet. The general instructions were to manipulate the deck 
of cards in the following fashion: 

1. Shuffle the deck and look over each card quickly. 

2. Sort the competency cards into three piles based upon the criteria 
of pile A containing those competencies which you feel are the 
most important for a teacher of graduate level courses to have. 
Pile B should include those competencies which you feel are not 
quite as important, and pile C those which are least important. 

3. Record the competency card n\imber on the score sheet in either 
column A, B, or C depending upon which pile it was placed in. 

The same decks of competency cards were also given to the faculty 
with the sa,:.e general instructions except that they were to use 
their level of adequacy as the criteria for separation. Piie A 



would be those cornpetencies in which they as a teacher of graduate 
level courses felt they were more than adequate to teach. Pile B 
were those In which they felt they had at least minimal adeauacy, 
and pile C were those in which they felt they were inadequate to 
'i^each. An added requirement for the faculty was that they also 
take each pile separately and rank order the items in that pile 
according to strength of adequacy. The responses on their score 
sheets therefore represent a ranlc ordering of all the items in 
three categories of adequacy, Ihis involved method of rank ordering 
was initially attempted by the students but was abandoned due to 
many tied ranks between which they felt discrimination was impos- 
sible. This was discovered by administering the Q-sort to a small 
pilot sample of student- 3. These sample students voiced negative 
responses toward the Q-sort task since it was forcing them to 
differentiate between items which they felt were eaual. Since 
there are argu-:ents in the literature such as Block(1956) and 
Jones (195^>) as to the benefits of Forced vs. Unforced sorting 
procedures, it was decided to allow Unforced sortings; as it was 
more agreeable for the students. 
Analysis 

For each item, the mean and the freouencies of the students' 
responses for the three categories of importance could be graphically 
depicted with a histogram (see Appendix B)# For each item, the 
mean of the students* responses could easily be determined by 
giving scores of 1,2,3 to piles A,B,C respectively. The lower 
the mean, the greater an items importance. Inspection of the 
histogram and a rank ordering of the means both revealed the 
relative imoortance of each item to the students. The responses 
of the teachers per item could also bo depicted through a histogram. 
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The means, however, did not have to be determined because if at 
least one teacher had an expertise in a particular fielc^ and was 
teaching this field then there was no gap. Gaps appeared if a 
particular competency had recieved a high rating of importance 
by the students but there were no faculty members which gave it 
a rating of at least minimal adeauacy. By this method it could 
be seen where the competencies desired by the students were not 
within tlifi repetoire of adequate competencies reported by the 
faculty, 

A more complete method of analysis would be to comoute a 
modified rank order correlation to reflect the degree of consensus 
among the students involved. A possible result of this would be 
to identify clusters of correlations from a matrix which would 
Indicate different groupings of students (Kerlinger, I973). 
By this method it could be seen if perhaps there are two opposite 
student orientations of importance such as "proactical application 
competencies" vs. "theoretical knowledge competencies". If this 
were the case then an item may get highly rated by one group 
and very low by another which would result in a misinterpreted, 
compromised mean. The recognition of differing groups would be an 
aid in reforming the curriculum and faculty to either include 
both orientations equally in courses or have two separate courses. 
If student inI;ercorrelations are homogenous which wou]d denote 
a fair amount of agreement among the students, then merely rank 
ordering the means of their responses will represent a reasonable 
estimate of the amount of importance placed upon a particular 
competency. 

Pinal inspection of the results completes the evaluation 
of/ this aspect of a program. If there are gaps then they will' 
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hnve been recognized. Prom this point it can then be recorrKaended 
that certain faculty members increase their present levels of 
particular .competencies, or that new or adjunct faculty be hired 
that have adequate competencies to fill positions not covered by 
the present faculty. Another interpretation of the results can 
show if the emphasis of the present curriculum is being placed 
upon the wrong areas ^ This would suggest a realTignment toward 
areas consir'ered more important by students. For example, in this 
study it was found that students in Counseling and Guidance 
rated "research and techniques of Blo-f eedback" as rather Important, 
but ther were not any classes offered in this area even though 
there were two qualified instructors on the present faculty. The 
obvious suggestion to be made is that there be a course concerning 
Bio-feedback. The other results of this study were basically 
positive for the teachers reported expertise in most of the 
fcompetencies rated important by the students. In the few areas 
that the teachers did not have expertise there were highly qualified 
adjuncts which taught this specialty. 

In conclusion, it appears that this particular use of the 
Q-sort technique is a productive tool in educational evaluation. 
First it can be determined which comoetencies should be within 
the range of the faculty. Then it can be shown through a histogram 
which competencies are considered imoortant by the students. 
It can also be shown if the data analyzed from the students 
represents a homjgenous group or not by intercorrelating each student* 
responses with each other. The resultant intercorrelation matrix 
can be inspected for groupings of students who correlate well 
among themselves but not with others* The level of adequacy for 
each competency for each teacher can be determined and compared 
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with the results of tho student data. Inspection of these comparison 

can reveal gaps between what the students feel Is Important 

for the faculty to be competent In and what the faculty feel they 

are actually competent In. Thus an educational program can be 
« 

evaluated in terms of the specli'ic competencies of its faculty. 
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Appendix A 

Examples of Items comprising the Q-sert for Counseling and Guidance 
Tqv which teachers in this field should know about or know how to do, 
1» Abnormal behavior and common disorders such as neuroses, 

psyc^hoses, orgariic conditions, etc, 
2, The physiological processes of behavior, 

3» Approaches and applications of marriage and family counseling, 
U* Various theoretical approaches to the explanation of social change 
5* Theories, terminology, and research in the learning process, 
6, Theory end principles involved in operant conditioning 

and contingency management, 
7# Methods of diagnosis and treatment of learning disabilities, 

8, Actual experience with the practical aspects and procedures 
involved in counseling, 

9. In depth analysis of those factors which put particular 
children to a disadvantage in our present public educational 
system, 

10, Principles of descriptive and inferential statistics, 
11m The physical and mental stages of human development, 

12, Techniques and practical apolications of behavior modif icationi> 

13, Tne various approaches and methodologies of counseling 
including research evidence and current issues, 

ll^.. Actual testing of children 3-8 years old to assess learning 
styles and diagnose potential problems, 

15, Aspects of verbal and non-verbal interpersonal communication, 

16, Actual experience with the practical aspects and procedures 
involved in counseling, 

17, Use of Bio-feedback as a research and theriapy technique, 
18« This card may be filled in and placed if desired, 
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Appendix B ; ' j 

i ■ i ' 

Example of histogram representation; 
oi students responses to item number 
J-o on the Counseling, and Guidance 
vi"Sort» 

1 i ! 

1- very Important 
2"minimal importance' 
3-not important. 

•i^A^^-J^'-, ^""^"^^^ expi^riende with thi 
practical aspects and procedures 
involved in counseling. 
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